ASYMPTOTIC ENUMERATION OF PERMUTATIONS AVOIDING GENERALIZED 

PATTERNS 



SERGI ELIZALDE 

Abstract. Motivated by the recent proof of the Stanley-Wilf conjecture, we study the asymptotic 
behavior of the number of permutations avoiding a generalized pattern. Generalized patterns allow 
the requirement that some pairs of letters must be adjacent in an occurrence of the pattern in the 
permutation, and consecutive patterns are a particular case of them. 

We determine the asymptotic behavior of the number of permutations avoiding a consecutive pattern, 
showing that they are an exponentially small proportion of the total number of permutations. For some 
other generalized patterns we give partial results, showing that the number of permutations avoiding 
them grows faster than for classical patterns but more slowly than for consecutive patterns. 

1. Introduction 

One of the most important breakthroughs in the subject of pattern-avoiding permutations has been 
the proof by Marcus and Tardos ^H] of the so-called Stanley-Wilf conjecture, which had been open for 
over a decade. This is a basic result regarding the asymptotic behavior of the number of permutations 
that avoid a given pattern. It states that for any pattern a there exists a constant A such that, if a n (cr) 
denotes the number of er-avoiding permutations of size n, then a n {a) < A™. The notion of pattern 
avoidance that this result is concerned with is the standard one, namely, where a permutation is said to 
avoid a pattern if it does not contain any subsequence which is order-isomorphic to it. 

In UD, Babson and Steingrfmsson introduced the notion of generalized patterns, which allows the 
requirement that certain pairs of letters of the pattern must be adjacent in any occurrence of it in the 
permutation. One particular case of these are consecutive patterns, which were independently studied by 
Elizalde and Noy For a subsequence of a permutation to be an occurrence of a consecutive pattern, 
its elements have to appear in adjacent positions of the permutation. 

Analogously to the case of classical patterns, it is natural to study the asymptotic behavior of the 
number of permutations avoiding a generalized pattern. This problem is far from being understood. It 
follows from our work that for most generalized patterns the number of permutations avoiding them 
behaves very differently than in the case of classical patterns. In this paper we determine the asymptotic 
behavior for the case of consecutive patterns, showing that if a is a consecutive pattern and a n (a) denotes 
the number of permutations of size n avoiding it, then linin^oo y a n (a)/n\ is a positive constant. For 
some particular generalized patterns we obtain the same asymptotic behavior, and for patterns of length 3 
the problem is solved as well. However, the general case remains open, and it seems from our investigation 
that there is a big range of possible asymptotic behaviors. For some generalized patterns a of length 4 
we give asymptotic upper and lower bounds on a n (a). 

The paper is structured as follows. In Section|21we introduce the definitions and notation for generalized 
pattern avoidance. We also mention some generating function techniques that will be used in the paper, 
as well as previous results regarding consecutive patterns. In Section|3]we give the exponential generating 
functions for permutations avoiding a special kind of generalized patterns, extending the results from 11 . 
In Section 01 we study the asymptotic behavior as n goes to infinity of the number of permutations of size 
n avoiding a generalized pattern, solving the problem only in some cases. In Sectional we give lower and 
upper bounds on the number of 12-34- avoiding permutations, and in SectionH3we obtain a similar result 
for the pattern 1-23-4. Finally, in Section [7| we discuss some open problems and further research. 

2. Preliminaries 

In this section we define most of the notation that will be used later on. We start introducing the 
notion of generalized pattern avoidance. 
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2.1. Generalized patterns. These patterns, which were introduced by Babson and Steingrimsson [3], 
extend the classical notion of pattern avoidance. We will denote by S n the symmetric group on {1, 2, ... , n}. 
Let n, m be two positive integers with m < n, and let 7r = "K\iX2 • • • 7r„ e 5„ be a permutation. A gener- 
alized pattern a is obtained from a permutation o\02 ■ • ■ c m € <?m by choosing, for each j = 1, . . . , m — 1, 
either to insert a dash - between <jj and <7j+i or not. More formally, a — <J\£\02£i ■ ■ ■£ m -i^m, where 
each £j is either the symbol - or the empty string. 

With this notation, we say that 7r contains (the generalized pattern) a if there exist indices i\ < %2 < 
. . . < i m such that 

(i) for each j = 1, . . . , m — 1, if Sj is empty then = ij + 1, and 

(ii) p{'Ki x 'Ki % •••7Ti m ) = C1O2 ' ' ' a mi where p is the reduction consisting in relabeling the elements 
with {1, . . . , m} so that they keep the same order relationships they had in ir. (Equivalently, this 
means that for all indices a and b, ni a < TTi b if and only if a a < o~b-) 

In this case, tt^tt^ ■ ■ ■ iXi m is called an occurrence of a in ir. 

If 7r does not contain a, we say that ir avoids cr, or that it is a-avoiding. For example, the permutation 
ir = 3542716 contains the pattern 12-4-3, and it has exactly one occurrence of it, namely the subsequence 
3576. On the other hand, ir avoids the pattern 12-43. 

Observe that in the case where a has dashes in all m — 1 positions, we recover the classical definition of 
pattern avoidance, because in this case condition (i) holds trivially. On the other end, the case in which a 
has no dashes corresponds to consecutive patterns. In this situation, an occurrence of cr in 7r has to be a 
consecutive subsequence. Consecutive patterns were introduced independently in jllj . where the authors 
give generating functions for the number of occurrences of certain consecutive patterns in permutations. 
Several papers deal with the enumeration of permutations avoiding generalized patterns. In jjj , Claesson 
presented a complete solution for the number of permutations avoiding any single 3-letter generalized 
pattern with exactly one adjacent pair of letters. Claesson and Mansour |Kj (see also |17p did the same 
for any pair of such patterns. In ^U], Elizalde and Mansour studied the distribution of several statistics 
on permutations avoiding 1-3-2 and 1-23 simultaneously. On the other hand, Kitaev investigated 
simultaneous avoidance of two or more 3-letter generalized patterns without dashes. 

All the patterns that appear in this paper will be represented by the notation just described. In 
particular, a pattern a = a\02 • ■ • & m without dashes will denote a consecutive pattern. We will represent 
classical patterns by writing dashes between any two adjacent elements, namely, as <Ti-(T2" ' ' ' _cr m- 

If a is a generalized pattern, let S n (a) denote the set of permutations in iS ra that avoid a. Let 
c* n (o~) = |<S n (cr)| be the number of such permutations, and let 



be the exponential generating function counting er-avoiding permutations. 

2.2. Labeled classes and exponential generating functions. Here we recall some basic machinery 
for exponential generating functions that will be used later. We direct the reader to ^21 f° r a detailed 
account on combinatorial classes and the symbolic method. Let A be a class of labeled combinatorial 
objects and let |£| be the size of an object ( E A. If A n denotes the objects in A of size n and a n = \A n \, 
then the exponential generating function, EGF for short, of the class A is 



In our context, the size of a permutation is simply its length. 

There is a direct correspondence between set-theoretic operations (or "constructions" ) on combinatorial 
classes and algebraic operations on EGFs. Table ^ summarizes this correspondence for the operations 
that are used in the paper. There "union" means union of disjoint copies, "labeled product" is the 
usual cartesian product enriched with the relabeling operation, and "set" forms sets in the usual sense. 
Particularly important for us is the construction "boxed product" A = B a *C, which corresponds to the 
subset of B *C (the usual labeled product) formed by those pairs in which the smallest label lies in the 
B component. Another similar construction is the "double boxed product" A — B s * C, which denotes 




n>0 
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the subset of B * C formed by those pairs in which both the smallest and the largest label lie in the B 
component. 



Construction 




Operation on GF 


Union 

Labeled product 
Set 

Boxed product 
Double boxed product 


A = BUC 
A = B*C 
A = 11(B) 
A = B a *C 
A = B m *C 


A(z) = B(z) + C(z) 

A(z) = B{z)C{z) 

A(z) = exp(B(z)) 

A{z) = $*{i t B(t))-C(t)dt 

A{ Z ) = j;j y a {^B(t))-C{t)dtdy 



Table 1 . The basic combinatorial constructions and their translation into exponential 
generating functions. 



2.3. Consecutive patterns of length 3. For patterns of length 3 with no dashes, it follows from the 
trivial reversal and complementation operations that a„(123) = a„(321) and a„(132) = a„(231) = 
a„(312) = a„(213). The EGFs for these numbers are given in the following theorem of Elizalde and 
Noy which we will use later in the paper. The symbol ~ between two sequences indicates that they 
have the same asymptotic behavior. 

Theorem 2.1 ([TTJ). We have 

a / ^ V3 e z/ ' 2 , , x 1 

Ai23{z) = — t= -, A 132 {z) 



2co S (fz + f)' ^ W l-ge-«lHi 

Their coefficients satisfy 

a„(123) ~ 7l • (pi) n • n\, a n (132) ~ 72 • (p 2 ) n ■ n\, 

where pi = 7i = e 3v/ ^ 7r , {P%)~ 1 * s unique positive root of Jq e~ l l 2 dt — 1, and 72 = eS P2 ^ I 2 , 
the approximate values being 

pi = 0.8269933, 71 = 1.8305194, p 2 = 0.7839769, 72 = 2.2558142. 

Furthermore, for every n > A, we have 

a„(123) > a„(132). 

3. Patterns of the form 1-a 

In this section we study a very particular class of generalized patterns, namely those that start with 
1-, followed by a consecutive pattern (i.e., without dashes). 

Proposition 3.1. Let a — o\Oi ■ ■ ■ G Sk be a consecutive pattern, and let \-o~ denote the generalized 
pattern 1-(cti + 1)((T2 + !)••• (ofc + 1). Then, 



A 1 - a (z) = exp A a (t)dt 

Proof. Given a permutation 7r, let m\ > TO2 > ■ • ■ > m r be the values of its left-to-right minima (recall 
that TTi is a left-to-right minimum of n if iTj > 71^ for all j < i). We can write ir = m\W\miWi ■ ■ -rrirWr, 
where each Wi is a (possibly empty) subword of 7r, each of whose elements is greater than mi. We claim 
that 7r avoids 1-a if and only if each of the blocks Wi (more precisely, its reduction p{uii)) avoids the 
consecutive pattern a. Indeed, it is clear that if one of the blocks Wi contains a, then rrii together with 
the occurrence of a forms an occurrence of 1-a. Conversely, if ir contains 1-a, then the elements of n 
corresponding to a have to be adjacent, and none of them can be a left-to-right minimum (since the 
element corresponding to T' has to be to their left), therefore they must be all inside the same block Wi 
for some i. 



4 



SERGI ELIZALDE 



If we denote by A the class of permutations avoiding a, then, in the notation of Tabled the class of 
permutations avoiding 1-er can be expressed as 

Il({z} D *A), 

where {z} D *„4 corresponds to a block rriiWi, with the box indicating that the left-to-right minimum has 
the smallest label. The set construction arises from the fact given a collection of blocks rriiWi, there is 
a unique way to order them, namely with the left-to-right minima in decreasing order. The expression 
Ai- a (z) — exp(f* A a {t) dt) follows now from this construction. □ 

Proposition 13 . 1 1 also appears independently in a preprint of Kitaev |15|. 

Example. The only permutation avoiding a = 12 (resp. a = 21) is the decreasing (resp. increasing) 
one. Therefore, by Proposition 13. II 



^1-23(2) = Ai- 32 {z) = exp yj e l dt 

the EGF for Bell numbers, which agrees with the result in 0. 

Example. For the consecutive patterns 132, 231, 312 and 213, the generating function for the number 
of permutations avoiding either of them is given in Theorem 12. II f which follows from [111 Theorem 4.1]). 
Now, by Proposition 13. II we get the following expression: 



A1-243 (z) = ^.1-342(2) = Ai- i2 3,{z) = Ai- 324 (z) = exp 



dt 



la l-f*e~ u2 / 2 du 



Example. The EGF for permutations avoiding 123 or 321 is also given in Theorem l2.ll Proposition l3.ll 
implies now that 

, \ a ^ f V3 [ z e'/ 2 dt \ 

^1-234(2) = A 1 -i3 2 {z) = exp — / —= . 

\ Jo cos(^t + £ ) J 

Combined with the results of Proposition ^. II gives expressions for the EGFs A\- a {z) where a has 
one of the following forms: 

a = 123 ■■■k, 

er = fc(Jfc-l)...21, 

a = 12- • -ar (a + 1), 

a = (a + l)ra(a- 1) • ■ -21, 

cr = k(k - 1) • • • (fc + 1 - a) r' (k - a), 

a = (k - a) t' (k + 1 - a)(k + 2 - a) • • • k, 
where fc, a are positive integers with a < k — 2, r is any permutation of {a + 2, a + 3, ■ • • , k} and r' is 
any permutation of {1, 2, • • • , k — a — 1}. 

4. Asymptotic enumeration 

Here we discuss the behavior of the numbers a n (a) as n goes to infinity, for a given generalized pattern 
a. We use the symbol ~ to indicate that two sequences of numbers have the same asymptotic behavior 
(i.e., we write a n ~ b n if lim, woo |^ = 1), and we use the symbol <C to indicate that a sequence is 
asymptotically smaller than another one (i.e., we write a n <C b n if lim n _ >00 ^ = 0). 

Let us first consider the case of consecutive patterns. 

Theorem 4.1. Let k > 3 and let a € Sk be a consecutive pattern. 
(i) There exist constants < c, d < 1 such that 

c n n\ < a n {a) < d n nl 

for all n > k. 
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(ii) There exists a constant < w < 1 such that 

lim ' 



Note that c, al and u> depend only on a. Compare this result with the conjecture of Warlimont [2*T] 
that for any consecutive pattern a there exist constants 7 > and w < 1 such that a n (o~)/n\ ~ 71/7™. 

Proof. The key observation is that, for any consecutive pattern cr, 

, , , . f m + n 
(1) a m+ „(cr) < a m (cr)a„(cr) 

V n . 

To see this, just observe that a er-avoiding permutation of length m + n induces two juxtaposed <r-avoiding 
permutations of lengths m and n. 
By induction on n > k one gets 

a m+n (a) < d m ml d n n\ I J = cT +n (m + n)! 

for some positive d < 1. 

For the lower bound, let r = pioxoicfi) be the reduction of the first three elements of a. Clearly 
S n (j) Q S n {o~) for all n, since an occurrence of a in a permutation produces also an occurrence of r, 
hence a n (r) < a n {o~). But the fact that a G £3 implies that a n (a) equals either a ra (123) or a n (132). In 
any case, by Theorem 12 .11 we have that 

a n (a) > a„(132) > c n n\ 

for some c > 0. 

To prove part (ii), we can express Q as 

(cr) a m (o-) an(cr) 
(m + n)! — ml n\ 

and apply Fekete's lemma (see |2(JI Lemma 11.6] or \V2\ ) to the function n\/a n {a) to conclude that 

lim n ^oo (y—j^^j exists. Calling it w, then part (i) implies that w < 1 and u> > limn^oo f — ^i 32 - 1 ^ = 
0.7839769. ' □ 

In order to study the asymptotic behavior of a n (a) for a generalized pattern a we separate the problem 
into the following three cases. Assume from now on that k > 3 and that a is a generalized pattern of 
length k. We use the word slot to refer to the place between two adjacent elements of a, where there can 
be a dash or not. 

• Case 1. The pattern a has dashes between any two adjacent elements, i.e., a = 0^-02-- ■ ■ -<Tfe. 

These are just the classical patterns, which have been widely studied in the literature. The asymptotic 
behavior of the number of permutations avoiding them is given by the Stanley- Wilf conjecture, which 
has been recently proved by Marcus and Tardos |18j . after several authors had given partial results over 
the last few years [Tl l2l 151 iffi] . 

Theorem 4.2 (Stanley-Wilf conjecture, proved in |18p. For every classical pattern a — o\-o~i-' • • -&k> 
there is a constant A (which depends only on a) such that 

a„(cr) < A™ 

for all n > 1 . 

On the other hand, it is clear that a n (a) > a n {p{o~i-o~2-o~3)) = C„ ~ — L=4", where C n denotes 
the n-th Catalan number. As shown by Arratia , Theorem 14.21 is equivalent to the statement that 
linin^oo ^/a n (cr) exists. The value of this limit has been computed for several classical patterns: it is 
clearly 4 for patterns of length 3, it is known ^U] to be (k— l) 2 for a — 1-2- ■ • • -k, it has been shown 0] to 
be 8 for a = 1-3-4-2, and it has recently been proved by Bona |o| to be nonrational for certain patterns. 
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• Case 2. The pattern a has two consecutive slots without a dash (equivalently, three consecutive elements 
without a dash between them), i.e., a = • • ■ OiOi+\Oi+2 

Proposition 4.3. Let a be a generalized pattern having three consecutive elements without a dash. Then 
there exist constants < c, d < 1 such that 

c n n\ < a n (a) < d n n\ 

for all n > k. 

Proof. For the upper bound, notice that if a permutation contains the consecutive pattern axa^cr^ ■ ■ ■ au 
obtained by removing all the dashes in a, then it also contains a. Therefore, a n (a) < a n (aia20~g, ■ ■ ■ a^) 
for all n, and now the upper bound follows from part (i) of Theorem 14.11 

For the lower bound, we use that a n (a) > a n (p{o~iO~i+\o~i+2)) > c< n (132) > c n n\, where <7j(7i+ia'i+2 are 
three consecutive elements in a without a dash. □ 



• Case 3. The pattern a has at least a slot without a dash, but not two consecutive slots without dashes. 



This case includes all the patterns not considered in Cases 1 and 2. The asymptotic behavior of a n {a) 
for these patterns is not known in general. The case of patterns of length 3 is covered by the following 
result, due to Claesson [xj. Let B„ denote the n-th Bell number, which counts the number of partitions 
of an n-element set. 

Proposition 4.4 ( jxj). Let a be a generalized pattern of length 3 with one dash. 

(i) Ifae {1-23,3-21,32-1,12-3,1-32,23-1,3-12,21-3}, then a n {a) = B„. 

(ii) If a e {2-13,2-31,31-2,13-2}, then a n (a) = C n . 

It is known that the asymptotic behavior of the Catalan numbers is given by C„ ~ ^3= 4". For the 
Bell numbers, one has the formula 



where A(n) is defined by A(n) ln(A(n)) = n. Another useful description of the asymptotic behavior of B, 
is the following: 

lnB„ /lnhn 
— m n — in m n + (J 



n \ Inn 

This shows in particular that S n -C B n -c c n n\ for any constants 8, c > 0. 

For patterns a of length k > 4 that have slots without a dash, but not two consecutive slots without 
dashes, not much is known in general about the number of permutations avoiding them. It follows from 
Cases 1 and 2 that 

S n < a(a\-a 2 a k ) < a n (a) < a(cria 2 • • • cr fe ) < d n n\ 

for some constants 6 > and d < 1. Clearly, if a contains one of the patterns in part (i) of Proposition l4~4l 
then the lower bound can be improved to B n . However, determining the precise asymptotic behavior of 
a n (a) seems to be a difficult problem. In the rest of the paper we discuss a few partial results in this 
direction. 

The next statement is about permutations of the form 1-cr. 
Corollary 4.5. Let a be a consecutive pattern, and let 1-cr be defined as in Provosition VJ .1\ Then, 



n — >oo \ nl / n — >oo 



Proof. By Proposition ^. ll we know that A 1 _ (T (z) = exp (J* A a (t) dtj . Since the exponential is an analytic 
function over the whole complex plane, we obtain that Ai- a [z) has the same radius of convergence as 
A a (z), from where the result follows. □ 
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5. The pattern 12-34 

The next proposition gives an upper and a lower bound for the numbers a n (12-34). Given two formal 
power series F(z) = X] n >o fn z ™ an d G{z) = ^2 n>0 g n z n , we use the notation F(z) < G(z) to indicate 
that /„ < g n for all n, and F(z) <C G(z) to indicate that f n <C g n . 

Proposition 5.1. For k > 1, let 




5(z)-^6 fc (z) + ^c fc (z). 
fc>i fc>i 

Then 

e s{z) < A 12 . 34 (z) < e SM+e*+*-i. 

If we write e s ^ = ^2lnTj an d e^ 2 ^ 6 + z ~ 1 = J2 u n^r to denote the coefficients of the series giving 
the lower and the upper bound respectively, then the graph in FigureQshows the values of y ct n (12-34)/n! 
for n < 13, bounded between the values \Jl n jn\ and y/u n /n\ for n < 120. The two horizontal dotted 
lines are at height 0.7839769 and 0.8269933, which are lim^oo y/a n (a)/n\ for a — 132 and a = 123 
respectively, given by Theorem l2.ll From this plot it seems conceivable that lim^oo y/a n { 12-34) jn\ = 0, 
although we have not succeeded in proving this. 
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Figure 1 . The first values of y/a n ( 12-34) Jn\ between the lower and the upper bound 
given by Proposition 15. II 
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Note that the lower bound, together with the fact that S(z) 3> e z — 1 (which follows from the definition), 
shows that A\2-za{z) > e S(z ) 3> e e _1 , which means that a n (12-34) ^> B„, that is, the number of 12-34- 
avoiding permutations is asymptotically larger than the Bell numbers. 

Proof. Let tt be a permutation that avoids 12-34. This means that it has no two ascents such that the 
second one starts at a higher value than where the first one ends. We can write tt = BQa\B\a2B2a^B^ • • • , 
where a\ and the element preceding it form the first ascent of tt, 0,2 and the element preceding it form the 
first ascent such that 02 < ax, a 3 and the element preceding it form the first ascent such that 03 < 02, 
and so on. By definition, Bq is a non-empty decreasing word whose last element is less than a\, and each 
Bi with i > 1 can be written uniquely as a sequence Bi — ii>i,oWi,i^i,2 • • • Wi,n for some Ti > 1 (ji can be 
if Wi t o is nonempty) with the following properties: 

(i) each Wij is a decreasing word, 

(ii) for j > 1, Wij is nonempty and its first element is bigger than a.£, 

(iii) the last clement of each Wij is less than at, 

(iv) the last clement of Bi is less than a,+i. 

These properties ensure that tt avoids 12-34 (since no Bi has an ascent above a,), and that the decom- 
position is unique. 

Ideally we would like to use this decomposition to find a generating function for the numbers a n ( 12-34). 
Unfortunately, the structure of the decomposition is a bit too complicated to find an exact formula. 
Instead, we will add and remove restrictions to simplify this description, which allows us to give lower 
and upper bounds respectively. 

To find an upper bound, we will count permutations of the form tt = BQa\B\a2B2a^B^ ■ ■ ■ , where the 
Bi and a$ satisfy the properties above, except for the requirement (iv) that the last element of each Bi 
has to be less than aj+i. Omitting this requirement we are overcounting permutations, and thus we get 
an upper bound. The first step now is to find the EGF for a block Ki of the form a,iBi, where Bi satisfies 
properties (i), (ii) and (iii) from above. 

Let us first assume that u^o is empty, that is, Bi = tw^iu^a • • • Wi, ri . We compute the EGF for 
Ki = aiBi where r$ is fixed, by induction on r^. If ri = 0, then we have that Ki — <Zj, so the EGF is 
60(2) := z. If Ti = 1, then Ki = a^iu^i, where w^i is a decreasing word starting above and ending 
below it. The EFG for is e z . Now, to incorporate the condition that the largest and the smallest 
labels of Ki lie in w^i, we use the double boxed product construction described in Section l2~2l A double 
derivative is needed to mark the two special elements. We get that the EGF for such a block is 

/ / t (^2 e ')^ d2/ = / / tet(itdy = i z - 2 ) eZ + z + 2 = b i(z). 

Let now = 2. The case in which both the largest and the smallest label of Ki = Oiio* iii;* 2 are contained 
in Wi t 2 corresponds to the EGF 

(2) ££M0 

If we write each Wij as wf^w^^, separating the elements above and below <Zj {wfj and w~ ^ respectively), 
then the largest element of Ki can be either in or in w\ 2 , and the smallest element of Ki can be 
either in w~ 1 or in w~ 2 . Thus, all the possibilities are obtained from the case counted by the EGF J2J) by 
permuting the upper and lower parts of the u>i t \ and Wi t 2 in the four possible different ways. It follows 
that the EGF for Ki when fj = 2 is 

rz rv 

4/ / b l (t)e t dtdy= (z - 3)e 2z + 4ze z + z + 3 = b 2 (z). 
Jo Jo 

In general, if bk-i(z) is the EGF for the case r, = k — 1, then the EGF for the case r, = k is given by 

b k (z) = k 2 [ f bk-i(jt)e*dtdy. 
Jo Jo 
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It is straightforward to check that the functions b k {z) defined in the statement of the proposition satisfy 
this recurrence. 

The case where u>^o is nonempty can be treated similarly. Now we have Bi = wo.iW^itf;^ • • ■ Wi,n- If 
Ti — 0, the EGF for aiWoj is cq(z) := e z — 1 — z (since the block has at least 2 elements). If r$ = 1, then 
a block of the form aiWo^Wi^x can be obtained from the case where the largest and the smallest element 
are in Wi.\ by permuting wq^ and w~ 1 if necessary. This yields the EGF 

f z f v ( d 2 \ e 2z 5 

2 jf J c (t) dtdy=— + 2(1 - z)e z -z-- = c x {z). 

In general, for nonempty Wi t o, if Ck-i(z) is the EGF for the case r% = k — 1, then the EGF for the case 
fj = k is given by 

c fc (z) = fc(fc + 1) / / Ck-ii^^dtdy. 
Jo Jo 

This is the recurrence satisfied by the functions Ck(z) defined in the statement of the proposition. 
The generating function for a set of blocks Ki — aiBi of the form just described is 

exp ^ b k (z) + ^ c fe (z) = exp(S(z) +z + e z -1-z). 

\k>0 k>0 J 

From such a set there is a unique way to form a sequence a\Bia2B2a^B^, ■ ■ ■ where a± > 0,2 > > ■ ■ • . 
Finally, we multiply by e z to take into account the initial decreasing segment Bq of the permutation 
tt = BoaiBia2B2Ci3B3 • • • , again relaxing the condition that its last element should be smaller than 01. 
This gives the upper bound e z exp(5(z) + e z — 1) = exp(S*(z) + e z + z — 1). 

Now we use a similar reasoning to obtain a lower bound. We have seen that bk(z) counts blocks of the 
form aiWi t \Wi^2 ■ • ■Wi,k^ where each Wij is a decreasing word starting above a% and ending below it. If 
k > 1, using the notation w^k = w tk w 7k ^° separate the elements that are bigger than from those that 



+ 



are smaller, we can move the last part of the block to the beginning and write Li :— w~ k a,iWi^Wi2 ■ ■ ■ w. 
Similarly, a block of the form aiWifiWisWi.2 • • 'Wi,k like the ones counted by Ck(z) with k > 1 can be 
reordered as L\ := w~ k aiWiflWi t iWi_2 ■ • ■ wf k . The EGF that counts sets of pieces of the forms given by 
Li and L\ is 

exp + Cfc ( z ) = exp(5(z)). 

yfc>i k>i J 

Ordering the pieces of such a set by decreasing order of the Oj, the sequence that they form by juxtaposition 
is a 12-34-avoiding permutation. Besides, no such permutation is obtained in more than one way by this 
construction. However, notice that not every 12-34-avoiding permutation is produced by this process, 
hence this construction gives only a lower bound. □ 

The decomposition of 12-34-avoiding permutations given in the proof of Proposition 15.11 can be gen- 
eralized to permutations avoiding a pattern of the form 12-er. If a — o\G2 ■••Cfc £ 4 is a consecutive 
pattern, 12-cr denotes the generalized pattern 12-(cti + 2) (02 + 2) • • • (<Tk + 2). 

Any permutation tt that avoids 12-cr can be uniquely decomposed as tt = BQa\B\a2B2a^B^ • • • , where 
cti and the element preceding it form the first ascent of tt, a.2 and the element preceding it form the first 
ascent such that 02 < esi, 03 and the element preceding it form the first ascent such that 03 < 02, and 
so on. Then, by definition, Bq is a non-empty decreasing word whose last element is less than ax, and 
each Bi with i > 1 can be written uniquely as a sequence Bi = WifiUi t x ^1,1^,2^1,2 ■ 1 ■ Ui iri Wi, ri for some 
i'i > 1 {fi can be if w^q is nonempty) with the following properties: 

(i) each ifjj is a decreasing word all of whose elements are less than a,, 

(ii) each Uij is a nonempty permutation avoiding a, all of whose elements are greater than aj, 

(iii) Wi t j is nonempty for j > 1, 

(iv) the last element of Bi is less than a^+i. 

From this decomposition the following result follows immediately. 
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Proposition 5.2. If a, r are two consecutive patterns satisfying A a (z) — A T {z), then A 12 - cr (z) = 

A 12 - T {z). 

The structure of 2 l-a- avoiding permutations (defined analogously) can be described using the same 
ideas, and it is not hard to see that the following result holds as well. 

Proposition 5.3. If a is a consecutive pattern, then Ai2- a (z) = A2\- a {z). 

6. The pattern f-23-4 

Similarly to what we did for the pattern 12-34, analyzing the structure of permutations avoiding 1-23-4 
we can give lower and upper bounds for the numbers a„(l-23-4). Let C exp (z) := Sn>o*-'™'ll ^ e * ne 
EGF for the Catalan numbers. 

Proposition 6.1. We have that 

\ £ e 2ey - 2 dy - | < Ax-aa^z) < C cx P(e z - 1). 

Writing \ J* e 2&v ~ 2 dy — § = X^nfn" anc ^ C oxp (e 2 — 1) = J2 u n7tf to denote the coefficients of the 
series giving the lower and the upper bound respectively, then the values of \Jl n jn\ and \Ju n jn\ for 
n < 90 are plotted in Figure [3 bounding the values of \/a n (l-23-4) jn\ for n < 11. 
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Figure 2. The first values of ^/a n (l-23-4)/n! between the lower and the upper bound 
given by Proposition 16. II 

Note that the lower bound implies that a n ( 1-23-4) 3> B„, since e 2e ~ 2 ^> e e _1 . 

Proof. Let n be a permutation that avoids 1-23-4. Let a\ > a 2 > a% > ■ ■ ■ > a r be the left-to-right minima 
of 7r, and let 6i > 62 > 63 > • ■ ■ > 6 S be its right-to-left maxima (recall that iti is a right-to-left maximum 
of 7T if Tij < TTi for all j > i). Then, marking the positions of the left-to-right minima and right-to-left 
maxima, we can write n — C1W1C2W2 ■ • • c r + s _iw r + s _ic r + s , where Cj € {eti, 02, ... , a r , b\, 62, ... , b s } for 
all i (in fact the number of c; 's could be less than r + s if some element is simultaneously a left-to-right 
minimum and a right-to-left maximum). Note that c\ — a\ and c r + s = b s . Now, the condition that tt 
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avoids 1-23-4 is equivalent to the fact that each Wi is a (possibly empty) decreasing word. Indeed, if there 
was an ascent inside one of the Wi, then together with the closest left-to-right minimum to the left of Wi 
and the closest right-to- left maximum to the right of itfj, it would form an occurrence of 1-23-4. On the 
other hand, it is clear that if all to, are decreasing, then no such occurrence can exist. 

We use this decomposition to obtain upper and lower bounds for a n (1-23-4). Let us first show the 
lower bound. For that we count only a special type of 1-23-4-avoiding permutations, namely the ones 
where all the left-to-right minima come before all the right-to-left maxima. Such a 7r can be written as 
7r = a\Wia2W2 ■ ■ ■ a r w r bi , w r +\b2W r +2 • • • Wr+s-ib s , where for 1 < i < r the elements of the decreasing 
words Wi have values between at and b\, and for r<i<r + s— 1 the elements of Wi have values 
between a r and bi+\. The EGF for the part aiWia,2W2 ■ ■ • a r -iWr-i is e e , since it is an arbitrary 
1-23-avoiding permutation (see the example following Proposition ^. 1|1 . Similarly, the EGF for the part 
w r+ ib2iv r +2 ■ ■ ■ w r + s -ib s is also e e _1 (it can be viewed as a set of blocks of the form uv+i&i+i, each one 
contributing e z — 1, arranged by decreasing order of the 6j's). The decreasing word w r contributes e z . 
Now, to get the EGF for the whole permutation a\W\a2W2 • ■ ■ a r w r biw r +ib2W r +2 ■ ■ ■ Wr+s-ib s we use the 
boxed product construction to require that the biggest element of the block is b\ and the smallest one is 
a r . The EGF that we obtain is 



—t)e e - 1 dtdy=- '~ 2e 
dt 2 i 



_1 I ft*) e * ( -J ) *° ~ dtd V = o / ( e - 



which gives a lower bound for the coefficients of ^1-23-4(2). 

To find the upper bound, consider first permutations of the form 7r = c\W\C2W2 ■ ■ ■ c r ^ s ~iw r + s -ic r + s 
where all the Wi are empty. Such permutations, where every element is either a left-to-right minimum or 
a right-to-left maximum, are precisely those avoiding 1-2-3, which are counted by the Catalan numbers. 
Thus, the EGF for such permutations is C exp (z). 

The next step is to insert a decreasing word Wi after each a. If Ci is a left-to-right minimum, we 
require that the elements of Wi are bigger than a, so the EGF for the block awi is e z — 1. We omit the 
requirement that the elements of Wi have to be smaller than the nearest right-to-left maximum to the 
right of this is why we only get an upper bound. Similarly, if Cj is a right-to-left maximum, we require 
that the elements of Wj are smaller than Cj, so the EGF for the block CjWj is also e 2 — 1. We also omit 
the requirement that after the last right-to-left maximum there is no decreasing word. Replacing each a 
for a block CiWi as just described translates in terms of generating functions into substituting e z — 1 for 
the variable z in C exp (z). This gives the stated upper bound. □ 

The upper bound given in the above proposition yields the following corollary. 

Corollary 6.2. We have that 

lim r"< 1 - 23 - 4 >y , "=o. 



Proof. The power series C exp (z) can be bounded by 



C cxp (z) < V 4™— =e 4z , 



n>0 



which converges for all z. Therefore, so does C cxp (e z — 1), which is an upper bound for ^1-23-4(2). The 
result follows now from the fact that if Y] f n z n is an analytic function in the whole complex plane, then 
lim„_ +00 \fj^x — (see ^3 Chapter 4] for a discussion) . □ 

If a = o~\o~2 • ■ ■ G Su-2 is a consecutive pattern, let 1-a-k denote the generalized pattern l-(ai + 
1)(<72 + !)■•• (cfe-2 + l)-k. The decomposition of 1-23-4-avoiding permutations given in the proof of the 
above proposition can be generalized to permutations avoiding any pattern of the form 1-a-k. 

Any permutation ir that avoids 1-a-k can be uniquely decomposed as ir — c\W\C2W2 ■ ■ ■ c m _iu> m _ic m , 
where the C4 are all the left-to-right minima and right-to-left maxima of 7r, and each Wi is a permutation 
that avoids a, all of whose elements are bigger than the closest left-to-right minimum to its left and 
smaller than the closest right-to-left maximum to its right. 
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Using exactly the same reasoning as in the proof of Proposition l6.ll we obtain the following lower and 
upper bounds for the numbers a n (l-o~-k). 

Proposition 6.3. Let a € Sk-2 be a consecutive pattern, and let 1-a-k be defined as above. Then. 

e 2j$A„(t)dt+y dydu < Ax-v-uiz) < C cxp ( f A a (t)dt 

Corollary 6.4. With the same definitions as in the above proposition, 

a n (l-a-k) \ a n (a) x 



lim 

n— >oo 



lim 

n — *oo 



Proof. The upper and lower bounds for A\-cr-h{z) given in Proposition ^) . 31 are analytic functions of A a (z), 
since essentially they only involve exponentials and integrals. Therefore, A 1 _ ._j ; (z) and A a (z) have the 
same radius of convergence, hence the limits above coincide. □ 

Finally, the following proposition is an immediate consequence of the structure of 1-cr-fc-avoiding 
permutations discussed above. In particular, it implies that ^1-23-4(2:) = ^1-32-4(2;). 

Proposition 6.5. If a, r are two consecutive patterns in Sk~2 satisfying A a (z) = A T (z), then Ai- a -k(z) = 

A\- T -k{z). 

7. Other patterns 

In Section[S]we have proved that a n (l-23-4) ^ B n and that a„(l-23-4) <C c n n\ for any constant c > 0. 
For the pattern 12-34, we showed in Section [5] that the analogue to the first statement holds as well, and 
the second one seems to be true from numerical computations. It remains as an open problem to describe 
precisely the asymptotic behavior of a n (o~) for these two patterns, and for several remaining generalized 
patterns of length 4. 




Figure 3. The first values of y/a n (a)/nl for several generalized patterns a. 



In Figure[3]we have plotted the initial values (connected by lines) of the sequences \/ u n (p) j n\ for other 
cases that appear to have some interest. The two dotted lines at the bottom of the graph correspond to 
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the sequences yC^Jnl and y /l 'B n /n\, which are known to tend to as n goes to infinity. The two dashed 
lines that start at the same point (around 0.941) and tend to a constant correspond to the sequences 
Y / a n (132)/n! and ^/a n (123)/n!, for which their limits are known by Theorem l2.1l to be 0.7839769 and 
0.8269933 respectively. Among the lines starting at 1, the two dotted ones correspond to the patterns 
1-23-4 (the lower line) and 12-34 (the upper line) discussed in the previous sections. 

Of the two solid lines, the one below corresponds to the pattern 3-14-2. This pattern has a special 
interest because all of its subpatterns of length 3 are among those in part (ii) of Proposition 14.41 Since 
it does not contain any of the patterns in part (i), we cannot say that a n (3-14-2) > B„ for all n. In 
fact, comparing the slopes in Figure it seems quite plausible that a n (3-14-2) grows more slowly than 
B„, and proving this is an interesting open question. The other solid line in the plot corresponds to the 
pattern 13-24, for which we do not know the asymptotic behavior either. 

This paper is the first attempt to study the asymptotic behavior of the numbers a n (a) where a is an 
arbitrary generalized pattern. Despite the fact that we have been unable to provide a precise description 
of this behavior in most cases, we hope that our work shows the intricateness of the problem and the 
amount of questions that it opens. The main goal of further research in this direction would be to give 
a complete classification of all generalized patterns according to the asymptotic behavior of a n (a) as n 
goes to infinity. 

Another interesting open problem is to find the value of lim„_ >00 ^/a n (a)/n\ for patterns a in Case 
2, for which this limit is known to be a constant. The analogous problem for patterns in Case 1, namely 
finding linin^oo y/a n {a) for classical patterns er, is a current direction of research as it remains open for 
most patterns as well (see 01 E])- 
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